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Abstract. We show that there can be no finite list of conditional independence relations 
which can be used to deduce all conditional independence implications among Gaussian random 
variables. To do this, we construct, for each n > 3 a family of n conditional independence 
statements on n random variables which together imply that X1JLX2, and such that no subset 
have this same implication. The proof relies on binomial primary decomposition. 



1. Introduction 

A fundamental question about the nature of conditional independence is whether or not there 
exists a finite set of conditional independence relations, from which all general conditional inde- 
pendence implications can be deduced. This problem was resolved in the negative by Studeny 
[6], who showed that there can be no such finite characterization. To prove this, he exhibited 
infinite families of conditional independence implications, using the multi-information function 
and properties of the submodular cone. 

Studeny's work leaves open the question of whether or not it is possible to find such a finite 
axiom characterization over restricted classes of random variables. For instance, if we assume 
that all random variables are binary, there are more independence relations that hold. Perhaps it 
is possible that among the many new independence relations there can be a finite axiom system. 

In this note, we show that there is no finite characterization in the regular Gaussian case, by 
exhibiting an infinite family of conditional independence implications which cannot be deduced 
from any other conditional independence implications. The main result of this note is: 

Theorem 1.1. Let X ~ AA(/n,S) with £ positive definite. Suppose that X satisfies the condi- 
tional independence constraints: 

X1ILX2IX3, X2-LLX3IX4, . . . , X n _iALX n \Xi, X n ALXi\X2 

with n > 4 . Then X also satisfies the marginal independence constraints: 

X1ALX2, X2ALX3, . . . , X n _iALX n , XiALX n . 

However, if X satisfies only a subset of the first CI constraints, it may not satisfy any of the 
second set. 

The proof relies on three simple ideas. First, we use the fact that conditional independence 
in the Gaussian case is an algebraic restriction on the covariance matrix S. So our collection of 
conditional independence constraints gives an ideal I n in R[S]. Next, we compute the primary 
decomposition of I n . Since, in our case, I n will be a binomial ideal, we can exploit the results of 
[I] to determine the minimal primes of I n . Then we use Hadamard products (a trick we learned 
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from [5]) to show that one of these components does not intersect the cone of positive definite 
matrices and that the other component yields the desired conditional independence implications. 

2. The Proof 

Recall that a regular multivariate Gaussian X ~ S) is complete specified by its mean 
vector [x € W 1 and its symmetric positive definite covariance matrix S G PD n . First of all, we 
show that conditional independence corresponds to an algebraic constraint on the covariance 
matrix S. 

Proposition 2.1. Let A,B,C be disjoint subsets of [n]. Then the joint distribution for X 
satisfies the conditional independence constraint AALB\C if and only if the submatrix ^a\jc,bvjc 
has rank less than or equal to ifC. 

Proof. If X ~ AT(/U, X) follows a multivariate Gaussian distribution, then the conditional distri- 
bution of Xaub\Xc = x c is also Gaussian with distribution 

N (vAuB + ^AuB,C^c^c( x C ~ MC?)j ^AuB,AuB ~ SauB.C^c^C.AU-b) j 

(see, for example, [1, §B.6]). The conditional independence statement AALB\C holds if and only 
if (T,aub,aub-^aub,c^c]c^c,a^b)a,b = 0. The A, B submatrix of T,aub,Aub—^Aub,c'Sc,C^C,Aub 
is easily seen to be T,a,b — ^A,C^c c^C,B which is the Schur complement of the matrix 

v f^A,B £U,c\ 

2->AUC,BUC = v V 

\^C,B ^C,C ) 

Since Sec is always invertible (it is positive definite), the Schur complement is zero if and only 
if the matrix T,auC,buC has rank less than or equal to j^C. □ 

Thus, if M. = {A\ALB\\Ci, . . . , A m ALB m \C m } is a conditional independence model, we nat- 
urally get the conditional independence ideal 

CI M = {#C + 1 minors of S Au c,BuC I AALB\C € M) C R[E]. 

To determine which conditional independence statements a collection of independence state- 
ments imply, we investigate the primary decomposition of the ideal CIm, and figure out which 
components of CIm intersect the positive definite cone PD n . 

Define M n to be the cyclic system of conditional independence constraints 

M n = {liL2|3,2_lL3|4,...,n-l-U_n|l,n_U_l|2} 

and let I n = CI_M n t> e the ideal defining this cyclic model. Or goal is to compute the primary 
decomposition of this ideal. 

2.1. Minimal primes of lattice basis ideals. Let B be a finite collection of integral vectors 
that form a basis for a saturated lattice £CZ". The lattice basis ideal associated to B is the 
binomial ideal 

I B = (x u+ - x u ~ ueB^C K[x], 

where u = u + — u~ is the cancellation free representation of u as the difference of two nonnegative 
integral vectors, and x a = the monomial vector notation. A thorough study 

of binomial ideals was done in [2] and a simple combinatorial characterization of the minimal 
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primes of lattice basis ideals was discovered in [I] . Their combinatorial characterization will be 
of considerable use, because of the following: 

Proposition 2.2. The conditional independence ideal 

In = ( "33*^12 <7l3<^23) 044^23 <^24<734i • • • j &\\&n—\,n 

is a lattice basis ideal. 

Proof. Let Uj denote the exponent vector attached the conditional independence statement 
-Xj_2-LLXj_i That is, Uj = en + ej_2i-i — ej-i^ — ej_2i, where the indices are interpreted 
modulo n. These vectors are linearly independent because the vector en only appears in vector 
Uj. Furthermore, this shows that the vectors span a saturated lattice because the submatrix of 
the matrix M n = [ui; . . . ; u n ] with columns corresponding to the diagonal entries in E, is the 
identity matrix. □ 

Since I n is a lattice basis ideal, we can use the results from [1] to study the primary decom- 
position of I n . Let B = {ui, . . . ,Ufc} be a basis for a saturated lattice. We form the matrix 
M € Z kxn whose rows are the vectors Uj. From [2], it is known that the minimal primes of 
a lattice basis ideal are completely determined by the indeterminates that appear in them. In 
particular, let S C [m] be a collection indexing indeterminates. Then the prime ideal associated 
to this subset is the ideal: 

I S = (xi I i E S) + <|x u+ - x u uGB, supp(u) n S = 0^ : 

Thus, one must determine the sets S that give minimal primes. When S = 0, Is = Ib '■ Y\ 
is the toric ideal associated to the lattice C. This toric ideal is always a minimal prime of Ib- 
The main result of [3] is that the other minimal primes of the lattice basis ideal Ib can be read 
off from the sign patterns in the matrix M. 

A matrix M is called mixed if every row contains a positive entry and a negative entry. Let 
S be a subset of [m] indexing a possible minimal prime. After permuting rows and columns of 
M, and relabeling S, we can assume that S = {1, 2, . . . , t} and M has the form 

M 

where iV has no all zero rows. 

Definition 2.3. A matrix N is called irreducible if 

(1) N is a mixed s x t matrix with t < s and 

(2) one cannot bring N into the form 

N : 



N 


B ' 





D 



N' 


B' 





D' 



by permuting rows and columns where N' is a mixed s' x t' matrix with t' < s' and D' 
is a (s — s') x (t — t') matrix with t — t' > s — s' . 

Theorem 2.4. [4j A set of S C [m] yields a minimal prime Is of the lattice basis ideal Ib if 
and only if the associated matrix N$ is an irreducible matrix. 
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Now we will apply Theorem 12.41 to determine the minimal primes of the ideals I n . 

Lemma 2.5. For n > 4, the minimal primes of l n are the toric ideal Ia„ = In '■ Y\ a ij an< ^ ^ e 
monomial ideal (<Ji2, 023, • • • , o~i n ). 

Proof. For n = 4, this can be checked directly in Macaulay2 [3] or proven directly by a slight 
variation on the argument below. So assume henceforth that n > 5. 

The toric ideal Ijy n is necessarily a minimal prime of I n . We need to show that the only set 
of variables S that induces an irreducible decomposition of the lattice basis matrix is the one 
corresponding to the set of variables {012, 0-23, • • • , cri n }. First, we want to describe the lattice 
basis matrix M n , which we need to find irreducible decompositions of. Since the binomials 
(7ii(7i_i j i_2 — 0"i-i,iCj-2,i only have index pairs (i, j) with \i — j\ < 2 mod re, we only need to use 
3re columns. We separate these columns into three groups, depending on whether \i — j\ = 0, 1, 
or 2. For n = 5, M5 is a 5 x 15 matrix: 



M 5 = 



/ 


+1 














+1 


-1 











-1 














\ 







+1 














+1 


-1 











-1 





















+1 














+1 


-1 











-1 





















+1 














+1 


-1 











-1 







V 














+1 


-1 











+1 














-1 


/ 



In general, after reordering the columns in each block, we will have an identity matrix, a 
circulant matrix on the vector (+1, —1, 0, . . . , 0), and minus the identity matrix for the three 
blocks of size re. Note that the central circulant block, corresponding to the indeterminates 
ci2 ; o"23; • • ■ > o~in, is an irreducible submatrix as it is n x n, mixed, and no nonempty submatrix 
of it is mixed. This is the desired minimal prime we were seeking. It remains to show that no 
other subsets of the variables induce an irreducible decomposition of the matrix M. 

To see why there are no other irreducible decompositions, we can just count the number of 
nonzero entries in each column. Note that there are < 2 nonzero entries in each column. So 
suppose that TV were an irreducible matrix arising from choosing some subsets S of the variables. 
Each row of N must have at least one +1 and one —1 entry, since N is mixed. So if N is an s x t 
matrix, the number of nonzero entries is > 2s. On the other hand, since each column has at 
most 2 nonzero entries, we know that there are < 2t nonzero entries in N. Furthermore, since N 
is irreducible we have s > t, so such an N can only exist when s = t and every column used has 
two nonzero entries. This implies that only the indeterminates corresponding to the circulant 
submatrix of M can be among the variables associated to the minimal prime. However, we have 
already seen that this set of variables yields an irreducible submatrix, thus there can be no other 
minimal primes besides the two we have already found. □ 

2.2. Hadamard Products. The next step in the proof depends on analyzing the toric ideal Ia„ , 
and showing that the variety V(Ia„) does not intersect the positive definite cone. Ultimately, 
the basic idea comes from a fact about Hadamard products of positive definite matrices. 

Lemma 2.6. Let S and T be nxn positive definite matrices. Then the Hadamard product S*T 
defined by 

(S * T)ij = a ij T ij 

is also positive definite. 
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Proof. Let X = (Xi,...,X n ) and Y = (Yi, . . . ,Y n ) be random vectors both with mean 
and with covariance matrices E and T respectively, and suppose that XALY. Also, suppose 
that X and Y have n-dimensional support. Let X * Y be the new random vector X * Y = 
(XiYi, X2Y2, . . . , X n Y n ). Since X and Y both have mean zero, we compute the covariance V of 
X*Y as 

7 - = ely^x^] = EpQA^]E[y t y,] = a tjnj . 

Thus, r = E * T. Since X and Y have full dimensional support, so does X * Y and thus T, 
the covariance matrix of a random variable with full dimensional support, must be a positive 
definite matrix. □ 

Lemma 2.7. The variety V(Ia u ) does not intersect the positive definite cone. 

Proof. Given a permutation ir E S n and an n x n matrix E, let 7r(E) be the matrix obtained by 
simultaneously permuting rows and columns by n. Let tt denote the cycle (12 ■ ■ ■ n) and let E 
be a generic matrix. Let the matrix E ra be defined by the repeated Hadamard product: 

E n = E * vr(E) * ^ 2 (E) * • • • * ^ n " 2 (E) * 7r n_1 (E). 

If E is positive definite, then so is E n . In particular, 

det ((E n )i 3) i 3 ) > 0. 

This inequality is equivalent to 

n n 

i=i 1=1 

On the other hand, the lattice ker^ A n is spanned by the vectors Uj = ea+ei-2,i-i— ej_i ; j — e-i-2,i- 
In particular, the vector 

n n n 

i=l i=l i=l 

is in the lattice ker^ A n . This implies that the binomial 

n n 

W^ii - \\o-i-2,i 

i=l i=l 

belongs to the toric ideal I& n . This implies that any E G ^(^a„) satisfies 



n n 



II 11 

i=l i=l 

Thus, y(/ A Jnp J D n = 0. □ 

As the last tool in the proof, we need to show that if X satisfies some, but not all, of the 
conditional independence statements XiALXi + i\Xi + 2, then none of the independence statements 
Aj_LLAj + i need to be satisfied. This is explained in the following lemma. 

Lemma 2.8. Suppose that X ~ A/"(/x, E) and X satisfies all the cyclic conditional independence 
statements except for X n _iALX n \Xi. Then X need not satisfy any conditional independence 
statements of the form Xj_LLXj + i. 
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Proof. It suffices to exhibit a positive definite covariance matrix that satisfies all the cyclic 
conditional independence constraints except cri\<y n -l,n ~ (T l,n cr i,n— l that does not have any zero 
entries. To this end, let < a, e < ^. Define an = 1 for all i, o~i-\^ = a n ~ l+1 for i G [n], 
o~i-2,i = o, for all i € [n], and o~ij = e for all other values of Note that we take the indices 
cyclically modulo re, so o"i >n = a n . 

First of all, note that the £ defined in this way is positive definite because it is diagonally 
dominant. Furthermore, for all i except i = 1, we have 

and thus, X satisfies all the cyclic conditional independence statements except X n -\SLX n \X\. 
Finally, by construction, all elements of S are nonzero. □ 

Proof of Theorem \l.l\ Lemmas 1 2 . 5 1 and [2771 show that V(I n )r\PD n = V({ai2, 023 • • • > a in))^PD n 
which implies the desired implication of conditional independence statements (reinterpretting 
those polynomial constraints back into CI constraints). Lemma [2.81 shows that if even one of the 
re initial CI constraints is omitted, there exists positive definite covariance matrices that satisfy 
the indicated constraints and do not imply any of the marginal CI constraints XjlLXj+i. This 
completes the proof of the theorem. □ 

References 

[1] P. Bickel and K. Doksum. Mathematical Statistics. Vol 1. Prentice-Hall, London, 2001. 
[2] D. Eisenbud and B. Sturmfels. Binomial ideals. Duke Math. J. 84 (1996), no. 1, 1-45. 

[3] D. Grayson and M. Stillman. Macaulay 2, a software system for research in algebraic geometry. Available at 

http : // www . math . uiuc . edu/Macaulay2/ 
[4] S. Hosten and J. Shapiro. Primary decomposition of lattice basis ideals, Journal of Symbolic Computation 

29 (2000) 625-639. 

[5] F. Matiis. Conditional independences in Gaussian vectors and rings of polynomials. Proceedings of WCII 
2002 (eds. G. Kern-Isberner, W. Rodder, and F. Kulmann) LNAI 3301, Springer- Verlag, Berlin, Heidelberg, 
152-161. 

[6] M. Studeny. Conditional independence relations have no finite complete characterization. In Information 
Theory, Statistical Decision Functions and Random Processes. 1992, 377-396. 

Department of Mathematics and Society of Fellows, Harvard University, Cambridge, MA 02138 



